Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

πŸ€–πŸ§‘β€πŸ”¬ [Experiment] Add Terraform Plugin Framework Code Generator #2355

Draft
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

jrhouston
Copy link
Collaborator

Description

This PR adds tools to allow generating terraform-plugin-framework code for new resources using the Kubernetes OpenAPI specifications.

Acceptance tests

  • Have you added an acceptance test for the functionality being added?
  • Have you run the acceptance tests on this branch?

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccXXX'

...

Release Note

Release note for CHANGELOG:

...

References

Community Note

  • Please vote on this issue by adding a πŸ‘ reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

@jrhouston jrhouston changed the title Add framework code generator πŸ€– Add framework code generator Dec 4, 2023
Base automatically changed from add-framework-provider to main December 4, 2023 14:37
@abhinavgrover15
Copy link

@jrhouston When i do a go generate, it does not generate any resources

@randomswdev
Copy link

randomswdev commented Jan 2, 2024

Tried to generate some Kubernetes resurces, but the resulting code does not work.
Consider a generic Kubernetes resource, that includes a spec and a status. Use the generator to generate the schema and the model for such a resource.

The model includes a struct representing the status. When creating an instance of a kubernetes resource, usually the status is not include in its specification (it is optional and computed). So Terraform attempts to set it as unknown.
Because the status is a struct, the terraform provider framework logic is unable to set the unknown status and the provider fails. A custom type would be required.

Also the manifest tags associated to each field in the generated structures have case issues. If the kubernetes resource contains an attribute named MyDHCP, the manifest entry is set to MyDhcp, thus creating a mismatch during the marshal and unmarshal processes.

The terraform openapi generator generates the correct structures, with types and values for every SingleNestedAttribute. But it is unable to add the manifest attribute that allows to map the openapi specification of the kuberntes resource to the generated model.

@jrhouston is there any plan to improve this pull request to have the generator generate better code? Mapping from kubernetes client structure to the terraform models generated by the openapi generator is really tedious

@jrhouston
Copy link
Collaborator Author

jrhouston commented Jan 8, 2024

Thanks for pulling down the branch and trying this out @randomswdev. This is very much a draft where my goal was to generate a simple resource like ConfigMap. Using this for large resources with spec and status certainly won't work yet – I have a long todo list of things to fix to make this work.

The model includes a struct representing the status. When creating an instance of a kubernetes resource, usually the status is not include in its specification (it is optional and computed). So Terraform attempts to set it as unknown.
Because the status is a struct, the terraform provider framework logic is unable to set the unknown status and the provider fails. A custom type would be required.

Yep, a mechanism for supporting custom types is on my todo list.

Also the manifest tags associated to each field in the generated structures have case issues. If the kubernetes resource contains an attribute named MyDHCP, the manifest entry is set to MyDhcp, thus creating a mismatch during the marshal and unmarshal processes.

Yeah, this is annoying. I suspect we're going to add a mechanism in the openapi generator tool to give us the OpenAPI context for the attribute so we can access the field name. If we can't do that then likely we'll need a way of configuring a known list of acronyms so cases such as your DHCP example don't get mangled.

@jrhouston is there any plan to improve this pull request to have the generator generate better code?

Yup, the plan here is to make this generator work more generally and use it to migrate all the existing SDKv2 code to the plugin framework. After this we will look at breaking it out into it's own module that can be used to generate custom providers for Kubernetes.

Mapping from kubernetes client structure to the terraform models generated by the openapi generator is really tedious

Just out of interest, is there a specific use case you are dealing with that motivated you to try this PR?

@randomswdev
Copy link

Just out of interest, is there a specific use case you are dealing with that motivated you to try this PR?

We want to automate some k8s operations that include activities on custom resources. So we need terraform providers to interact with those CRDs.

We are looking for a fast and cheap way to develop these providers, and your generator appeared to be a quite easy solution.

Currently we opted for leveraging the OpenAPI generator with hand written code for mapping the generated models back and forth to the k8s API client structures.

If the future direction will be to extend the OpenAPI generator and the terraform plugin framework code generator to propagate "manifest" tags to the generated models, we will be able to automate also the mapping. And the need for custom code would be limited to very specific scenarios.

@jrhouston jrhouston changed the title πŸ€– Add framework code generator πŸ€–πŸ§‘β€πŸ”¬ [Experiment] Add Terraform Plugin Framework Code Generator Apr 28, 2024
appilon
appilon previously approved these changes Apr 29, 2024
go.mod Outdated
@@ -148,3 +150,5 @@ require (
sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
)

replace github.com/hashicorp/terraform-plugin-codegen-kubernetes => /Users/john/dev/hashicorp/terraform-plugin-codegen-kubernetes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remember to fix this once codegen repo is public

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants